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liBSTRiCT 

A group of high school self-reported dropouts and a 
group of simulated high school dropouts were compared with graduates 
in their participation rates^ efforts reguired for data collection^ 
and the overall quality of their responses in the National 
Longitudinal Study of the High School Class of 1972, The results 
indicate that dropouts are less likely to return mail questionnaires^ 
have lower retention rates in a longitudinal study, need more effort 
to trace their residence and to complete editing checks, and 
generally provide data of lower quality. The results also indicate 
that no substantial differences in data quality exist between those 
dropouts and graduates who were interviewed. It is thus suggested 
that the interview approach is preferable as far as data quality is 
concerned. The combined use of mail survey, extensive telephoning, 
and interviews would also be advisable. In either case, 
questionnaires should be simple and straightforward, i complicated 
and lengthy questionnaire would probably discouraqe dropouts from 
completinq the questionnaire. (Author/EVHf* 
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Abstract 



A group of high school self-reported dropouts and a group of 
simulated high school dropouts were compared with graduates in 
their participation rates, efforts required for data collection, and the 
overall quality of their responses in a longitudinal survey. Results 
indicated that dropouts were less likely to return mail questionnaires; 
had lower retention rates in a longitudinal study; needed more effort 
to trace their residences and to complete editing checks; and general- 
ly provided data of lower quality. Results mo indicated that no 
substantial differences in data quality existed between those drop- • 
outs and graduates who were interviewed. It is thus suggested that 
the interview ai^proach is preferable as far as data quality is con- 
cerned. A combination of mail survey, extensive editing phone calls, 
and interviews would also be advisable. In either case, quedtionnaires 
should be simple and straightforward. A complicated and lengthy 
questionnaire would probably discourage dropouts from completing 
the questionnaire. 



Foreword 



The National Longitudinal Study of the High School Class of 
1972, a survey initiated by and conducted for the National Center 
for Education Statistics, began in the spring of 1972 with over 1,000 
in-school group administrations of sUrvey forms to a sample of 
approximately 18,000 seniors. In the follow-up surveys, the sample 
was augmented by almost 5,000 additional students from sample 
schools that were unable to participate in the base-year survey. 

The data collected from the in-school and two follow-up surveys 
have been merged and processed. Results are being presented in a 
series of reports, designed to highlight selected findings in educa- 
tional, career, and occupational development. This report contains 
information about the differences between higlf school graduates and 
dropouts in participation rates, data collection efforts, and response 
quahty. 

Continuing follow-up requests for data from these individuals are 
planned through 1979 and perhaps beyond. This series of repeated 
observations will permit the examination of the relationships be- 
tween schooling, work, and other experiences to subsequent .career 
choices as well as educational and labor force participation of each of 
the selected individuals. Such information and the resultant analyses 
are important to those engaged in formulatir g legislative proposals 
and educational policy. 

Francis Corrigan, Director Elmer F, Collins, Chief 

Division of Multi-Level Longitudinal Studies Branch 

Education Statistics 
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Differences Between High School Graduates and Dropouts 
in Participation Patterns and Response Quality in Survey Studies 



1. BACKGROUND OF THE STUDY 



This study w£ts designed to seek answers 
to questions concerning the data collection 
methods intended for use in future surveys of 
the National Longitudinal Study (NLS). 
Although the resuRant information can have 
broader generalizations beyond NLS, a lyief 
description of the study background should 
be helpful in giving a better understanding of 
the problem 

. Th^ National Longitudinal Study is a con- 
tinuing project monitored and f)rimarily 
funded by the National Center for Education 
Statisttcs (NCES). This project represents a 
number -of agencies and offices within the 
Department of Health, Education, £ind Wel- 
fare with interests in the long-range effects of 
educational policy. The focus of the NLS is 
on the educational, vocational, and personal 
development of high school students, ana the 
personal, familial, social, institutional, • and 
cultural factors that contribute directly or 
indirectly to that development. The general' 
purpose is to establish a factual basis for veri- 
fying and refining federal policy concerned 
with maximizing individual access to educa- 
tional and vocational opportunity, with 
improvement of the general educational 
system as it impinges on young people, and 
with positive impact toward aiding young 
people to assume a productive, satisfy mg, and 
wholesome adult role iii society. A secondary 
purpose is to extend the general scientific 
knowledge of human development in the 
important years covering the transition from 
high school to adult careers. 

The NLS began with a stratified, two- 
stage probability sample of over 21,000 
students representing the graduating high 
school class of 1972, with follow-up to date 
in 1973, 1974, and 1976. For various reasons 
the 1972 study was lestricted to those indi- 
viduals who were high school seniors in the 
spring of 1972. The study design did not 
include high school dropouts, an especially 



important group if policy-oriented research is 
to deal, as it should, with the broad dynamics 
and wide variations of equal opportunity by 
race, class^and sex. 

As another projected part of the National 
Longitudinal '^tudy, substantial replication of 
the 1972 study is planned with a later cohort, 
probably the High School Class of 1980, with 
the important addition of including indi- 
viduals who had been members of that class 
but who left school before graduating.^These 
dropouts will be included so that their post- 
• school development may be compared with 
that of thobe who do graduate, and to broad- 
en the population'' base to which gener- 
alizations concerning educational effects can 
be applied. 

The inclusion of a dropout componentin 
the stydy, however, requires additional' con- 
siderations for data coUection methods 
because of the following questions: Are drop- 
outs more difficult to locate, less likely to 
complete certain types of instruments success- 
fuUy, and less cooperative than nondropouts? 
Previoifts research has not provided adequate 
empirical bases for ansivering these questions. 
References' can be drawn only from some 
related studies. ^ Foi example, it has been 
found that less educated people have higher 
nonresponse rates in mai' survey (Moser .& 
Kalton, 1972) but are the most likely to be 
located (Crider & Willits, 1973), and that less 
educated people are more likely to give 
contradictory re^onaec (Lenski & Leggett, 
1960). It has also been found that poor 
achievers in college have a higher nonresponse 
rate (Neuss, 1943), and that more non- 
respondents are from families of lower 
socioeconomic background (Vincent, 1964). 
None of these studies, however, have sygtem- 
aticaUy investigated the response patterns of 
high school dropouts ^ in comparison with 
graduates. It is within this context that this 
study was designed. 
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II. GENERAL APPROACH FOR THE STUDY 



«To answer the above questions, one could 
identify higi^ school dropouts and graduates 
throu^ school records, and then actually 
conduct a study to compare, for example, 
their participation rates and re^onse quality. 
This approach would be preferaBle to the 
approach used in this study. It would, how- 
ever, be very costly. 

The ^proach used in this study was to 
use available data from^the 1972 NLS study. 
In this study, there were 127 sample members 
who reported that they left high school with- 
out completion by the fall of 1973. This 
group of students, labeled as self-reported 
dropouts, constitutes a useful study group 
except for the following constraints: (1) the 
n (sample size) may be too small for national 
generalizations; and (2) they may not be 
typical high school dropouts considering their 
leaving school only a few months before 
graduation. They probably have higher 
achievement and different school attitudes 
than those who, hft school before tenth grade. ^ 

To compensate for the shprt'^omings of 
the self-reported dropouts, a group of drop- 
outs was simulated from the NLS sample. It 
was felt that a well-simulated group would 
yield as reliable estimates as an actual group 
and at a low cost. 

This study thus includes three groups for 
comparisons'. (1) self reported dropouts; (2) 



simulated dropo^; and (3) graduates. The 
first gtoup was ised in a sense to validate the 
second group; that is, their characteristics and 
survey response qualities provide a basis for 
evaluating how close the simulated dropouts 
are to the actual dropouts. The' simulation 
procedures are described in thB next section. 

One consideration in the conduct of this- 
study was that the information ^bout tracing 
effort and editing requirements was not in the 
data file. In addition, the data comprising the 
current NLS data files may not truly repre- 
sent the quality of the raw questiormaire data 
because student responses to some key 
questions in each survey were manually edited 
and checked by phone calls to respondents 
before being keyed into data files. Conse- 
quently, the observed differences betweep 
dropout and graduate groups regarding data 
quality, using the -data fdes only^ could be 
underestimates of actual diiferences. Hence, 
the use of the original or unedited data is 
desirable for comparisons on data quality. 
However, checking the original res^ onses 
nieant going back to microfilni records of 
original questionnaires which, if the entire 
NLS sample were involved,' would be ex- 
tremely expensive. Therefore, subsamples of 
100 cases each wer^ randomly drawn fro^n 
each of the dropout and graduate groups to 
obtain supplemental information for com- 
parisons on tracing effort, editing require- 
ments, ahd data quality. 



III. SIMULATION OF HIGH SCHOOL DROPOUTS 



The question addressed in this section 
is: Which individuals should be classified as 
simulated dropouts? Before answering this 
question, a description of the general profile 
of high school dropouts-Hhose students leav- 
ing school without completion-^ould be 
helpful. 

Recent estimates of the number of high 
school dropouts range generally Irom about 
20 to 30 percent of the school-age popu- 
lation, depending upon the group to whom 
the label is applied and the population base 
used. Variations by state and region are, of 



coarse, large. In any 'rase, all studies support 
Dentler and Warshauer's (1965) description of 
the model dropout as **a low school achiever, 
usually below grade level for his age. He is a 
member of a low-income family in which 
parents have low educational attainment." A 
national study conducted by the Survey 
Research Center (SRC) at the University of 
Michigan for the U.S. Office of Education 
found that whereas about 40 percent of the 
tenth grade students with IQs below 90 
dropped out of school, fewer than 10 percent 
of those H^th scores over 125 left school 



(Bachman, Green, & Wirtanen, 1971). In the 
same study, it was found that 40 percent of 
the students who were held back a grade level, 
versus 10 percent of those who remained in 
grade, subsequently dropped out of school. 
The relationship between academic grades and 
dn^pping out was even stronger. The SRC 
study also shows that whereas aJbout 30 per- 
cent of the tenth grade students in the lowest 
socioeconomic quartile dropped put of high 
school, fewer than 10 pferdept from the 
highest qusurtile dropped out!' 

bjropouts tend to' come fron^ rural areas 
and large cities. The dropout ra^e in rural 
afeas is almost twi<!;e as lai^e as that in the 
urban fringe (U.S. Bureau of Census, 1973). 
Dropputs also tend to have a stronger negative 
attitude toward school (e.g., school is ver^ 
boring) and to place less yalue on academic 
achievement. (Bachman, Green, & Wirtanen, 
1971). Dropouts also tend to be more rebel- 
lious^ in school than nondropouts. The SRC 
.^tudy estimated that nearly one-half^ of those 
students who '*often" engaged in rebellious 
behavior in $chool later became dropouts. 

^ In summary » dropodts are a group of 
students who are generally poor, low 
achievers, and behind in grade level. They 
have high alienation toward schoxjl and place 
little value on academic achievement. 

In light of what is kriown about the- 
characteristics of high school dropouts, the 
next step was to identify simulated dropouts 
from the 1972 study sample. The 127 self- 
reported dropouts were separated first from 
the sample members, and then simulated 
dropouts were ident'fied from the remaining 
sample members. For the purposes of this 
study, a simulated dropout was defined as a 
sample member who had self-reported high 
school grades befow C, was in the lowest 



quartile of SES,* and was not in any kind of 
post-secondary education by the fail of 1974. 
(High school grades were u^d bl'cause they 
reflect not only an individual's academic 
ability b«t also his motivation. Likewise, SES 
reflects family socioeconomic conditjion'and 
parental expectations. As discussed pre- 
viously, these variables are highly related to 
dropping but of school.) The rest of the 
sample members, by definition, were grad 
uates. Some o^ them, however, did not have 
information aboiit high scnool grades or SES, 
and thus were unclassifiable. This group of 
members was excluded from Comparisons. 
The f i n al numbers of NLS participants 
involved in this study were 127 self -reported 
dropouts, 980 simulated drop}outs, and 
15,771 graduates. 



* SES'-Avas based upon a composite of father's 
e ducation, mother's education, parental 
income, father's occupation, and a house- 
hold items index. Factor analysis revealed a 
common factor with approximately equal 
loadings for each of the five components. 
Missing components were imputed as the 
mean of the subpopulatioa of which the 
respondent Was a member, defined accord- 
ing to cross-classifications of race, high 
school program, and aptitude. The available 
standardized components, both imputed 
" and nonimputed, were averaged to form an 
SES when at least two aonimputed 
componepts were available. The continuous 
SES score was ther assigned to one of the 
quartiles on the basis of the weighted fre- 
quency distribution of the composite score. 
. The first quartile, the middle two quartiles, 
and the fourth quartile were respectively 
denoted as the low, middle, and high SES. 



IV. FURTHER DESCRIPTION OF DROPOUTS AND GRADUATES 



Tibltl. 

SOME BACKGROUNP CHARACTFUISTICS 
OF DROPOUTS ANO GRAOUATES 



• 

Diaricttfiitics 


Self- 
Reported 

Dropouts 


Simulated 

Dropouts 


Graduates 


— : z 

Ration 


% 






Northeast 


18.1% 


16.3% 


22.0% 


Mom Central 


29.9 


31.7 


27.1 ^ 








17 7 


West 


28.3^ 


122 


18.3 , 


Academic Ability* 








High 


0.3 


1.8 


25.4 


MiHHIii 

^ IfiiUUIv 


1 D.J 




47 1 


Low 


50.4 


65.7 


25.2 


-Unknown 


26 8 


8.4 


70 


5(nrinptf*nnnm ir RsM^lf nrniinH 
wvwiviSbUfiuiii II* Doc^yiuufiu 








" High 


10.2 


0.0 


24.9 


Middle 


52.0 


00 


50.7 


Low 


37.0 


100.0 


24.5 


High School Program 








General 


Dj.o 


40.0 




Acarfemic 


9.4 


8.3 


43.0 


J/oc-Tech 


32.3 


43.0 


23.8 


Sex 








Male 


b5.4 


58.8 , 


4a .9 


Female ^ 


34.6 


41.2 


51.1 


Race 








Black > 


14.2 


25.1 


1U 


Hispanic 




9.3 


4.3 


White 


ro.i 


56 5 


78.6 


Other 


7.8 


9.1 


5.3 


Type of Residence 








Commtinity 








Rural 


23.6 


30.7 


21 6 


Small City of TDwn 


^ ,18J 


25.2 


26.2 


Large City 


36.2 


25.8 


27.6 


Suburban 


15.0 


13.6 


22.3 


Unknown 


6.3 


4.7 


2.4 


H 


127 


980 


15,771 



The ability me^'sure was a composite score of four tests: 
Vocabulary, ReatfTng, Letter Groups, and Matherhatics. A 
factor analysis fevealed a general academic ability factor 
that was represented by an equally weighted linear 
romposita of these four standardized tests. The composite 
score was classiflad Into a low, middle, or high category 
corresponding to the first quartile, the middle two 
quartiles, or the fourth quartile. 



Dropouts (both self-re ported and 
simulated) and graduates were further 
examined with reFnect to their background 
and characteristics. As Shown in- Table 1, 
simula^d dropouts have a great similarity to 
self-iep6rted dropouts. Both groups had 
greater percentages of members from the 
South and North Central regions and from 
non-college-preparatory programs- as com- 
pared to the graduate group. Tliey also were 
composed of a higher proportion of non- 
whites and 'men and had lower acadeinic 
ability and S^S than graduates. Both self- 
reported and simulated drd||outs were moreT^ 
likely to come from rural areas or lai^c citi^9< 
than suburban areas. v * 

Prior research has shown that dropouts 
are generally more negative toward school 
than nondropouts (^.g., Bachman, Green, & 
Wirtanen, 1971). This was also true among 
the simulated and self-reported dropouts. As 
shown irr Table 2» a substantially higher 
percentage of dj^dpouts than graduates ex< 
pressed such negative- :ittitudes toward schoo) 
as **don*t feel part of i the school" and 
"parents aren't intereste^ in Iny \^ucation." 

Simulated dropouts, like jplfi-r^orted 
dropouts, were -older than"graduates. The 
average age levels in October 1972 were 



Table 2 ' . 

PERCENTAGE DF DRDPDUTS AND GRADUATES 
EXPRESSING NEGATIVE ATTITUDES 

TOWARD S6HDDL \ 





Attitude 


Self- 
Reported 
Dropouts 


Simulated 
Dropouts 


\ 

Graduates 


1. 


Plan to attain less than 










high school graduation. 


10.31% 


9.16% 


1J1% 




Caui^t'sare too hard. 


8.42 


4.61 


^ 196 


3. 


Parents aren't interested 










in my education. 




^ 27.7? 


13.^ 


4. 


Don't feel part of the 










school. 


17.03 


14.63 


9.87 


5. 


Find it hard to adjust 










to school routine. 


14.71 


829 


5.71 




N 


98 


962 15,415 
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18.87, 18,73, and 18.50, respectively, for • 
simulated dropouts', self-reported dropouts, 
and graduates. The data supported previous 
findings that dropouts t^nd t6\ be behind 
grade level (Dentier & WarshWr, 1965). 

fii sumipary, the simulated ^poats had' 
characteristics and attitikles geFfenftTy ^ob- 
served among actual high school dropouts. 
The profile seemed to agree with wha^ Vrould , 



be expected on the basis of previous research. 
However, it should be not<* J tjiat*the* simu- 
lated as Well as self-reported dropouts were 
beoiors during the NLS base-year survey." 
/Differences in survey data between dropouts 
and graduates would probably be greater than 
\those observed m thi^^ study because most 
. dtopauts leave " school Wfefore the twelfth 
graHe, 



VI ANAIYSES AND RESULTS 



A. D if f erences in Ov^all ' 
Partrcipation Rates . 

^ The first t|uestion addressed, in thi^ study 
isTifte dropouts 1^^ likely tlyui gractuiitert<^ 
participate in\a ^i^rvey? To answer this( 
question-, the percentage of members who, 
"partipipated in the base ye^ and the two; 
lollipw-iip stirveys Was ofomput^dfor the drop-y 
out \ aAd tljke graduate group$. / Hesuits jurfe \ 
presented in\able 3. In the base-year, survey,; 
the difference between Emulated dropbuts! 
and ' graduates was not substantial, but 'ttie 
self-reported drbpcJuts had a ^much lo^r 
participation (rate. It is possible tfiat those 
self -reported dropouts were absent o'' had 
already left school at the time the Student 
Questionnaire was administered in school in 
the spjrmg of 1072. Thus, their lower partic- 
ipation rate c^i^ld be due to the fact that 
some of them c^id not receive the ques- 
tionnaire. In the first follow-up survey, th^ 



simulated djropouts had a lower rate than 
graduates, and the 100 percent participatign 
^ rate of self-reports dropouts- was expected 
because only Ithbse members who responded 
y to the first follow::up questionnaire could 
have information for dropout classification. 

. In the second foUbw-up survey, the self- 
reported dropouts and graduates had si)nili*r 
{participation rates, but the simulated drop- 
outs h^ri a lower rate. The inconsistency 

.between the self^eported dropouts and 
simukt'-^^dropoufs could .be due to varying ' 
reasons. For example, all simulated dropouts 
were from ihe lowest qualtile of SES, and low 
$ES in^mbers tended to have lower partic- 

.ipatiotn, rates (e,g.; Vincent, 1964). It is also 
po^sjbte that .those self-reported dropouts 
were probably those menibersVho were more 
likely. t6 participate in the survey; otherwise, 
HHey couit^i have dropped' out of the first 
follow-up survey and would not be included 
in this group. \ 



Tables 



PARtlCrPATION RATESIN (ACH SURVEY 



^ ^ ' ^ J ' = 

Base Yean ^ FinttFottmv-Up S*cond Follow Up 

— t > 1 . ^ — . 

Self* Self- ' Salf- 
Reported Gnduat^ Reported ^l'"""^"' Griduitts Reported ^l'""'*^"' GiwIurtK 
Drbpouts "'""'"^ Dropouts """""^ Dropouts """'"^ 



Participants " 
Nonparticipants 



77.2?) 
22.8 



98.2% 
1.8 



97.7% 
2,3 



100.0% 87.6% 
0.0 12.4, 



94.5% 
5.5 



92J9% 852% 
7 14.8 



93.3% 
6.7 



Note: 

1. Sample n's - 127 self-reported dropouts; 980 simulated dropouts; and 15,771 graduates. 

2, Self-reported dropouts were K^entifled based upon responses to an education-status question in \\m first follow-up ques- 
tionnaire. 
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Although the results did not provide a 
cleai^ basis for ^evaluating whetlier or not' 
dropouts are less likely to participate in a 
•survey, they did sliow that dropouts had 
hi^er withdrawal rates. Th^ participation 
rate of simulated dropouts changed from 98 
percent h\ the base year to 85 percent in the 
second foUpw-up, while the participation rate 
of graduates changed from 98 to 9o percent. 
Tfi^ self-reported dropouts also showed a 7 
percent decline from the first to the second 
f611ow«up suirvey. (The base-year rate was not 
used for this. Comparison because the ques- 
tionnaire might not have been administered to 
some>of th^m because'of their absence.) This 
may create some biases in the long run if the 
same participation trend continues; that is, 
aftftr extensive mail and interview effort, the 
^survey samplp »3 comprised of fewer persons 
with high school dropout characteristics. 

B. Diifferences in Mail Return Ratesr 

Differences , in the overall participation 
rate may be masked by the extensive efforts 
to interview those who failed to return 
questionnaires by n^ail. To examine tliis 
possibility and for its own value, the follow- 
ing question was asked: Are dropouts less 
likely to respond by mail if they do partic- 
ipate? The answer to this question is clearly 
shown in Table 4. Dropouts, both self- 
Teported and simulated,, had a much lower 
^ mail return rate than graduates. In the first 
folfow-up survey, the mait>retum rates were 
about 39,-47, and 71 percent, respectively, 
for ' self-reported dropouts, simulated drop- 
out$, and graduates. Although the mail return 



rsrtes increased for each^ group in the second 
follow-up survey, substantial differences 
between dropouts and gra'duates stUl existed. 
Results clearly indicate a grater difficulty in 
collecting data from^droi^uts by mail quos- 
. tionnaires aHl^oi^gh they may participate in 
the study if intcr^^ews are conducted. 

Difforenoei in gracing Efforts 

Some previous studies have indicated that 
J[ess * educated people areX^ier to loteate 
probably because of their b^ing less mbbile 
(e.^., Crider & Willits, 1973)Nt^e present 
sXudy, however, dges Aot support tfti$ finding. 
Based upon a subsample of 100 pjptes from 
each group, and considering onlj^ the^^st^ 
follow-up survey, dropouts in general required . 
more tracing efforts /to uodate their mailing 
addresses than gradijfeites. jA* shown in Table ; 
5, the' numbers (here equal to percents) j 
requiring telephone tracing were 28, 34, and 
21, respectively, for sel|-r€fported dropouts, 
rimulated dropouts, and graduates. Furth^s 
more, among whose memlrers requiring tele- 
phone tracing, 39 percent (i.e., 11/28 X 100) 
of self -reported dropouts and 50 percent of 
simulated dropouts, as compar^^d to 29 
percent of graduates, had an unlisted phone 
or did not have a private phone, and thus tiiey 
required more indirect calls. Even among 
those "having private phones, dropouts needed 
more calls to complete the tracing. The 
average number of tracing calls was about 6.8 
(i.e., 115/17) for self-reported dropouts, 6.5 
for simulated dropouts, and 5.7 for graduates. 
In terms of telephone- costs, dropouts were 
more e^fpensive to trace than graduates. 



TiWs 4 \ 
PERCENTAGES OF MAI*. RETURNS AND INTERVIEWS 







First Follow-Up 






Second Follow-llp 






Self -Reported 

DroptiiiS 


Stmuleted 
Dropouti 


Graduetes 


Self ^Reported 
Dropouts 


Simulated 
Dropouts 


Graduates 


Mail RBturn 


39.37 


46.50 


71.30 


' 51.69 


61,56 . 


76.44 


Interview 


60.63 


53.50 


28.70 


48.31 

lb 


38.44 


24.58 


N 


127 


858 


14399 


118 


835 


14,712 



Note: No inteuiews were conducted in the base-year survey. 
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Tabit S 

TRACING EFFORTS FOR JROPOUTS AND GRADUATES 
(FIRST FOLLOW-UP) 



Sdf- 

Dropouts 

Dropouts 



Number of Participants 
R^uinng Tracing 28 

Having Private Phone 

Having No Private Phone 

Number of Participants 

Not Requinny Tracing^ 72. 



17 
11 



?4 



66 



17 
17 



21 



79 



15 
6 



Total Phone talis 190 275 J.28 

Having Private Phone 115 110 85 

Having No Private Phone 75 1 65 43 



100 



100 



100 



D. Differences in Weekly Postal 
Return Rates 

Another question addressed in this study 
is. Does it take a longer time to get responses 
from dropouts than graduates if they do 
return a mailed questionnaire? To answer this 
question, weekly and cumulative postal return 
rates using date of receipt as the basis were 
computed for the three groups. Results are 
presented in Tables 6 ^d 7, resooctively, for 
the first and second f It can be 

easily seen that there ^ :bo.dn*;ial differ- 

ence in postal return rates oetween dropouts 
and graduates in the first lew weeks. For 
example, dunng the first week after initial 
mailout of the First Follow-Up Questionnaire, 
about 28 percent of graduates returned the 
questionnaire as eompared to 14 percent of 
the simulated dropouts. Prop irtidnally, about 
twice as many graduates as simulated drop- 
outs returned the questionnaire in tlie first 

C f 



Table 6 

RELAi IVE FREQUENCY OF POSTAL RETURNS BY NUMBER OF 
WEEKS FROM THE FIRST QUESTIONNAIRE MAILING 

(FIRST FO LLOW UP) 

F req uency Cumulative F requency 

Self- Self- 
Weelfs Reported Simulated Reported Simulated 

Dropouts Dropouts Graduates Dropouts Dropouts Graduates 



1 


79 


136 


28.1 


7.9 


13.6 


28.1 


2 


2.4 


34 


10.6 


10.3 


19.0 


38.7 


3 


08 


2.1 


4.2 


11.1 


21.1 


^Z.9 


4 


0.8 


13 


3.3 


11.9 


22.4 


4r,2 


• 5 


24 


1.8 


2.4 


14.3 


24.2 


48.0 


6 


102 


5.0 


5 1 


24.5 


29.2 


53.7 


7 


5.5 


\ 36 


4.1 


r 30.0 


32.8 


57.8 


8 


lA 


1.5 


2.4 


32.4 


34.3 


60.2 


9 


0.0 


1.0 


1.3 


32.4 


35.3 


R1.5 


1U 


24 


0.6 


0.8 


34.8 


35.9 


62.3 


:i 


2.4 


0.3 


0.5 


37.2 


36.2 


62.8 


12 


0.0 


0.4 


0.2 


37.2 


36.6 


63.0 


.12 


2.2 


4.1 


4.4 


39.4 


40.7 


67.4 



H 127 930 15.771 127 980 15,771 



- second QuesttOhnaire matiout. 
•12 - more than 12 weeks. 
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Table? 

RELATIVE FREQUENCY OF POSTAL RETURNS BY NUMBER OF 
WEEKS FROM THE FIRST QUESTIONNAIRE MAILING 
(SECONO FOLLOW-UP) 



Frtquen.y Cumulttivt Frtquency 



Self- Stif- 
Weeks Reported Simulated Reported Simulated 

Oropouts Oropouts Graduates Oropottts Dropouts Graduates 



1 


18.1 


19.8 


25.6 


18.1 


19.8 


25.6 


2 


9.4 


9.5 


16.3 


27.?; 


29.3 


41.9 


3 


5.5 


4.9 


6.6 


33.0 


34.2 


48.5 


4 


1.6 


3.4 


4.6 


34.6 


37.6 


53.1 


- 5 


1.6 


1.5 


2.0 


36.2 


39.1 


55.1 


6 


0.0 


2.9 


3.2 


36 2 


42.0 


58.3 


7 


2.4 


24 


3.0 


38.6 


44.4 


61.3 


8 


0.8 


1.9 


1.2 


39.4 


45.3 


62.5 


9 


0.8 


0.5 


0.7 


40.2 


45.8 


63.2 


10 


0.8 


1.2 


1.Z 


41.0 


47.0 


64.4 


11 


0.0 


1.0 


0.8 


41.0 


48.0 


65.2 


12 


0.0 
7.0 


0.3 


0.4 


41.0 


48.3 


65.6 


>12 


4.1 


4.8 


48.0 


52.4 


70.4 



N 127 980 1S771 127 980 15,771 

^ a = ^ ^ 



' second questionnaire mailjur 
>12 - more than 12 weeks. 



four weeks during the first foUow-up survey. 
This ratio was shghtly reduced in the second 
follow-up survey. 

The cumulative frequency of postal 
returns for the groups was graphed in Figures 
1 and 2, respectively, for the first and second 
ioHow-ups. These curves depict the similarity 
in postal return patterns between self-rf port- 
ed and simulated dropouts. The curves help 
reveal the length of time required for ^h* 
group to reach a certain response rate. 'AH 
groups developed their total response rates 
rapidly in the first few weeks, and then 
leveled off. A slight jump in the sixth week is 
due to a second mailout. 

It can be seen that it took a much longer 
time for dropouts to reach the early graduate 
response rates. For example, it took about 
nine Weeks for simulated dropouts to reach a 
35 percent postal return rate whereas it took 
graduates less than two weeks to do so in the 



first follow-up survey. The return rates of 
dropouts after 12 weeks were still ^ess than 
the percentage of questionnaires retumeu L* 
graduates after only two weeks (38.7 per- 
cent). 

Although postal return rates for dropouts 
were slightly higher in the ^icond follow-up 
than in the first follow-up, the differences in 
questionnaire response data between dropouts 
and graduates were stilLconsiderable. After 12 
weeks of data collection efforts, the return 
rates for dropouts were still less than the rate 
for graduates at the three-week point. 

E. Differences in the Extent of Editing 

Eiefore being keyed onto tapes, all NLS 
questionnaire responses went through manual 
e diting to check inconsistencies and the 
completeness of key questions. Any respond- 
ents who failed these edit checks were called 
up to clarify or complete their answers. 
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14 



65" 



BO- 



SS 



50- 



45- 



L40- 



35- 



30- 
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Self Reported Dropout *• 
Simulateo Dropout ^ — 

Graduate • 



J L 



J L 



1 2 



7 



10 11 



t 



Delivery Weeks From First Mailing 
Note t indicates the second questionnaire mailout 

Figure 2. Cumulative Postal Retuhi Rate (Second Follow Up Survey) 
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Extensiveness ^ editing provides another 
basis Tor comparison between graduates and 
dropouts: Are dropouts more likely to fail 
edit checks than graduates? 

Of those 100 randomly selected cases 
from each group, 39 percent of self-reported 
dropouts and 41 percent of simulated drop- 
outs returned the First Follow-Up Ques- 
tionnaire by mail. However, 77 of the selected 
graduates responded to the mailed ques- 
tionnaire. Among these respondents, the 
percentage of dropouts who failed editing was 
about twice as large as that of graduates. The 
corresponding percentages for the three 
groups were 41, 42, and 22 percent. It is thus 
concluded that dropouts' questionnaires 
required a greater extent of editing (see Table 
8). 



TabltB 

PERCENTAGE OF MAIL RETURN QUESTIONNAIRES 
THAT FAILED EDITS 
(FIRST FOLLOW UP) 





Satf-Raportad 

Dropouts 


Simulated 
Dropouts 


Graduates 


Fail Edit 


40.82 


42.11 


22.08 


Pass Edit 


59.18 


57.89 • 


77.92 


N 


39 


41 


77 



F. Differences in Quality Measures 

It has been shown that dropouts are less 
likely to respond to a mailed quest' ->nnaire, 
take a longer time i' they do respi ,id to a 
questionnaire, and need more editing efforts 
than graduates The next question is: Do 
dropout and graduate participants provide 
data of similar quality? That is, how con- 
sistent, reasonable (e^., not out-of-range), 
and co'mplete are their responses? Do they 
foUow directions correctly? 

To answer the$i^ questions, dropouts and 
graduates were compared on several quality 
measures. The derivations of those nieasures 
are briefly described as foUows: 

1 . Inconsistency Index ( IS) 

This index represents the percentage of a 



set of consistency checks failed by an 
individual. The index was computed by the 
foUowing formula: 



IS = 



i-l.n 



100 



where Xj = 



0 if the respondent passed check 
i; 



1 if the respondent failed check 

n = number of consistency checks. 

For the first follow-up questionnaire, 
there were 94 consistency checks. For the 
second foUqw-up questionnaire, two incon- 
sistency indices were developed with Index 1 
calculated from 36 manual edit checks for 
key items, and Index .2 from 21 edit checks 
for non-key items. 

2. Out-of-Range Index (OR) 

This index represents the percentage of 
out-of-range responses for an individual's 
record and was computed as foUows: 

OR = N/D • 100 

where N = the number of items with an 
out-of-range code; and 

D = the number of items with 
responses. 

3. Routing Error Index (RE) 

This index represents the percentage of 
the routing questions that were ambiguously 
ans\vered by an individual (i.e., routing 
questions which were unanswered or an- 
swered in a manner which was inconsistent 
with the respondent's subsequent pattern of 
r e sp o n se ). The routing error index was 
computed as foUows: 

RE = N/D • 10( 

where N = the number of flagged routing 
questions; and 

D = the number of routing questions 
with responses other than legiti- 
mate skip. 
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/ 4. Incompleteness Index (IP) 

For a particular section of the ques- 
tionniEbpe, the incompleteness index represents 
the percentage of items ^^th invalid 
responses— that is, responses that are coded as 
errors or missing data (first follow-up codes 
93-98, second follow-up codes 94-98). The 
index was computed as follows: 

' IPi - N/D • 100 
where i = the particular section of the 
questionnaire; 

N = the number of items with invalid 
responses (responses with error or 
missing data codes); and 

D = the number of items with 
responses other than legitimate 
skip. ^ ^ ^ 

In the following analyses, dropouts and 
graduates were further classified by mode of 
data collection (i.e., mail versus interview). 
This was necessary because more dropouts 
than nondropouts were mterviewed, and 
interview data are assumed to be more 
complete than mail responses. 

The analysis results, based upon the 
released NLS data of the dropouts and 
graduates, are presented in Tables 9 and 10. 
All the measures are in a negative direction; 
thus, the higher measures mdicate poorer 
quality of the data. 

The general pattern exhibited in the tables 
is that d/opouts, both self -reported and 
simulated, tended to provide poorer quality 
data than graduates in their mail-returned 
questionnaires. They were more likely than 
graduates to proviHp inconsistent and 
incomplete responses, and to make more rout- 
ing errors (i.e., not following directions 
correctly). In tbe first follow-up, the dif-, 
f e rence between simulated dropouts and 
graduates was largest in quality measures of 
routing error and incompleteness in the 
education and work experience sections. In 
the second follow-up, the difference again was 



very high in routing error and incompleteness 
of the last three sections of the questionnaire. 

The magnitude of variance of quality 
measures, as represented by standard 
deviations, also indicates that dropouts were 
more heterogeneous than graduates with 
respect to data quality. 

Tables 9 and 10 also present the measures 
of the quality of data collected through inter- 
views. They did not show any substantial or 
consistent differences between dropouts and 
graduates. This seems to indicate that the 
quality of interview data is not dependent 
upon type of respondent. 

\s mentioned previously, responses to a 
few key items in the fijst and the second 
follow-up questionnaires were edited and 
corrected through phone calls if response 
inconsistencies occurred. This editing oper- 
ation may mask some differences between 
dropouts and graduates. To test this assump- 
tion, unedited data obtained from the random 
sample of 100 participants from^ach of the 
simulated dropout and the graduate groups 
were analyzed. Results, as presented in the 
Appendix, showed that basic difference" in 
the quality of mail-returiied data did exist 
between dropouts and graduates. Th^t is, if 
the unedited data were used in comparisons 
of data quality, the existence and magnitude 
of differences between dropouts and grad- 
uates would still be substantial; the editing 
operation has not masked the basic differ- 
ences. 

It soems reasonable to conclude that drop- 
outs will not .provide as good data as grad- 
uates through mail questionnaires. Dropouts 
are more likely to make routing mistakes and 
are less likely to complete the questipns, 
possibly because of poorer reading skills. If a 
mail survey is adopted in a study, the problem 
of missing or inconsistent data will be r re 
severe among dropouts, and thus may con- 
tribute some bias to analyses involving drop- 
outs. To insure data quality, interview 
procedures will be preferable for dropouts. 



12 



Tablt9 

MEANS AND STANDARD DEVIATIONS 3F DATA OUALITY INDICES 
(FIRST FOLLOW UP) 



Mail IntarvtfiM 



Dita Quality Index Reported Pfinulated RepOittd Simulattd 

Dropouts Dropouts Graduates DropoircS Dropouts Graduattt 



1. InconsistencY 2.30 

(3.72) 

2. Out-of Range 0.04 

(0 20) 

3. Routing Error 5.B2 

(8.05) 

T 

4. incompleteness in 
Section: ^ 

a. General [n|ormation 14.88 

(9.11) 

Education and 
Training 

c. Civilian Work 12.62 
Experience (22S8) 

d. Military Service 1.26 

<5.97) 

e. Information About 10.94 
the Past (21.43) 



116 US 

(1.99) (1.82) 

0.14 COB 

(0.97) (O.L.? 

5 &d . 3.28 

(6.39) (P.10) 



20.57 13J9 
(12.78) (25.36) 



20.10 11.34 

(27.78) (20.31) 

3.57 ' 1.47 

(16.02) ;i0.34) 

13.58 8.G0 

(24.08) i 16.94) 



0.82 0.69 0.80 

(1 13) (1.43) (1.08) 

GDI 0 02 0.04 

(0.11) (0.21) (0.38) 

2.43 2.17 2.St 

(4.64) (4.03) (3,6/) 



7.87 17.70 17.18 

(10.14) (26.84) (26.53) 

12.87 22.C6 23.72 

^25.74) (37.85) (32,63) 

3.53 15.32 15 25 

(8.65) (30.66) (30,56) 

3.34 2.90 2.61 

(13.43) (15^7) (14.79) 
% 

3.96 16.66 18.31 

M2.74) ; (3524) (34.78) 



77 459 ^4^76 



N 50 399 ie,B23 . 



Note: Figures in parentheses are standard deviations. 



35 86 40.1B ^ 24.44 

(35.52) (39.37) ' \26.37) 
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Table tO 

MEANS AND STANDARD OEVIATIDNS DF DATA QUALITY INDICES 
(SECDNO FOLLOW-UP) 
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Mail 






Interview 




Data Quality Index 


Siif' 
Reported 
Dropouts 


Simulated 
Dropouts 


Graduates 


Self- 
Reported 
Dropouts 


f^tmulated 
Dropouts . 


Graduates 


1. Inconsistency (Key 
Items) 


0.9O 
(2.17) 


0.46 
(1.67) 


0.29 
(1.02) 


0.07 
(0.37) 


0.12 
(0.64) 


0.17 
^(0.69) 


2. Inconsistency (Non- 
key Items) 


2.62 
(4.64) 


1.90 
(3.61) 


1.73 
(3.46) 


2.82 
(4.28) 


2.^1 
(4.45) 


J 3.31 
(5.97) 


3. Routing Error 


6.02 
(8,67) 


3.ZI 
(6.72) 


2.18 
(3.79) 


2M 
(4.42) 


2.27 
(3.08) 


2.09 

, {2 SB) 


4. Out-of'Range 


0.02 
(0.13) 


0.02 
"(0.10) 


0.02 
(0.38) 


0.00 
(0.09) 


0.01 
(0.10) 


0.01 
^ (0.14) 


5 Inrcinpfeteness in 
Section: 

a. General Information 


(7 22) 


2.18 
(7.58) 


1.31 
(531) 


12.70 
(27 19) 


18.06 
(31.47) 


14.95 
(29.23) • 


b. Education and Training 
Training 


27.69 
(35.39) 


20.38 
(37.06) 


15.85 
(24.26) 


18.75 
(33.72) 


24.73 
(37.06) 


24.13 
(34.72) 


c. Work Experience ' 


20.43 
(27.00) 


17.08 
(24.23) 


14.63 . 
(21.63) 


24.07 
(35.11) 


27.83 
(38.78) 


24.36 
(36.11) 


d. Family Status 


11.69 
(12.44) 


11.38 
(15.88) 


. 7.25 
fl1.48) 


19.89 
(37.29) 


25.74 
(41.53). 


^ 21.74 
(39.00) 


e. Military Service 


5.93 
(20.38) 


3.90 
(16.14) 


1.25 
(8.66) 


2.68 
(13i)0} 


5.62 
(19.16) 


2.76 
(13.94) 


f. Activities and 
Opinions 


8.48 
(13.09) 


5.91 
(12.04) 


3.32 . 
(7.61) 


20.33 
(39.18) 


25.45 
(42.51) 


24.77 
(39.85) 


N 


61 


514 


11,099 


57 


321 


3,613 



Note: 1. Figures in psrentheses are standard deviations. 

2. The comparison between Interview and mail responses is not legitimate because some questions were not 
asked in interviews due to the nature ^f the questions. 
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VL SUMMARY AND DISCUSSION 



The primary purpose of this study was to 
examine the problems m data collection from 
high school dropouts. A group of self- 
reported dropouts and a group of simulated 
dropouts were compared with graduates on 
their participation rate, tracing efforts, postal 
return delivery date, and the quality of their 
responses. Major findings are summarized in 
the foUowin^? paragraphs. 

Postal return ^'^s are lower among drop- 
outs. The difference between dropouts and 
graduates generally runs between 25 per- 
centage points in the first follow-up and 18 
percentage points in the second follow-up. To 
maintain a high participation rate, dropouts 
will require more phone or field interviews 
than graduates. Dropouts also need more trac- 
ing efforts to update their mailing address, 
partly due to their greater tendency to not 
return mailed materials. 

Dropouts are less likely than ^^duates to 
respond to a mail survey promptly. At the 
end of 12 weeks the return rates of dropouts 
were no higher than those of graduates after 
only two or three weeks. 

About 40 percent of the dropouts' 
questionnaires failed editing and required 
further phone calls to complete or correct 
their questionnaires. This was about twice as 
many as graduates, in addition, the overall 
quality of the data from dropouts is less 
satisfactory thaif that from graduates, drop- 
outs tend to provide more erroneous re- 
sponses (e.g., out-of-range, mconsistent, and 
incomplete responses) than graduates. 

It is reasonable to conclude that it is 
much more difficult to obtain quality data by 
mail from dropouts. In addition, the declining 
participation rate of dropouts through time 
may bias the sample In later foUow-ups. If the 
dropout component is included in a sample, 
additional efforts should therefore be taken 
to maintain the dropout participation and at 
the same time insure high quality data. 

The interview approach, either through 
phone or field interviews, is preferable for 
dropouts with respect to maintaining a high 
response rate and high quality of data. How- 
ever, a combination of mail survey and inter- 
view may prove to be cost effective. Mail 



ret am may be improved by making the 
questionnaire as simple and straightforward as 
possible. Many dropouts may have difficulty 
in reading, and thus difficult language, 
complex skip patterns, and a lengthy ques- 
tionnaire may confuse or discourage many 
dropouts from participating. To the extent 
possible, these barriers should be removed. In 
addition, mail return may be increased by 
some incentive mechanism. The offer of a 
$3.00 incentive in the third foUow-up survey 
will present an opportunity for testing the 
influence of incentives on dropouts and grad- 
uates when the data are available. It seems 
that the. motivation (willingness) to partic- 
ipate and the ability (i.e., reading skill) to 
complete a questionnaire are the two major 
factqrs in a mail survey. Steps to cope with 
these two major factors are advisable in future 
studies involving dropout components. 
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Appendix 

Means and Standard Deviations of Data Quality Indices 
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^ Apptndix 

MEANS AND SI ANOARO DEVIATIONS OF DATA QUALITY INDICES 
(UNEDITED FIRST FDLLOW UP DATA FRDM RANDOMLY DRAWN SAMPLES) 



Oati Quality Indtx 



Mail 



Simulatad 
Dropoim 



Graduates 



Intarvievtf 



Simulatad 

Dropouts 



Graduates 



Out-of Range 



0.03 
(0.16) 



0,04 
(0.221 



0.00 
(0.00) 



0.00 
(0.00) 



Routing Error 

Incompleteness in Section: 
ad General information 

b. Education and Training 



c. Civilian Work Experience 

d. Military Service 



e. Information About the 
Past 



5.16 
(4.86) 



19,57 
(1179) 

4832 
(38.67) 

26.05 
(29,50) 

2,84 
(11.79) 

12,05 
(19.36) 



3.51 
(5.52) 



15,09 
(10.48) 

21.87 

(22,20) 

10.87 
(18.58) 

2.86 
(15.82) 

8.25 
(19.51) 



2.30 
(4.39) 



14.70 
(25.09) 

15.09 
(32.13) 

l'>25 

1.93 
(11.76) 

13.91 
(33.04) 



2.47 
(3.22) 



13.71 
(24.Q2) 

19.76 _ 
(30.02) 

* 9.57 
(24.53) 

(0.00) 

16.86 
(30.73) 



N 



37 



77 



44 



21 



Note: No inconsistency index was computed because of the complexity of computer progrdm changes required and the 
insignificant number of changes in responses to the items involved in the computation. 



23 



A-2 



